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Use of Evidence and Evaluation in the 2014 Budget 


Since taking office, the President has emphasized the need to use evidence and rigorous 
evaluation in budget, management, and policy decisions to make government work effectively. 
This need has only grown in the current fiscal environment. Where evidence is strong, we 
should act on it. Where evidence is suggestive, we should consider it. Where evidence is weak, 
we should build the knowledge to support better decisions in the future. 

Agencies should demonstrate the use of evidence throughout their Fiscal Year (FY) 2014 
budget submissions. Budget submissions also should include a separate section on agencies’ 
most innovative uses of evidence and evaluation, addressing some or all of the issues below. 
Many potential strategies have little immediate cost, and the Budget is more likely to fund 
requests that demonstrate a commitment to developing and using evidence. The Budget also will 
allocate limited resources for initiatives to expand the use of evidence, including but not limited 
to approaches outlined below. Agencies may include these initiatives in their submission at the 
guidance level or with proposed addbacks. 

1 . Proposing new evaluations . As in 201 1 and 2012, OMB invites agencies to propose new 
evaluations. Areas of potential focus may include the following: 

• Low-cost evaluations using administrative data or new technology: As explained in 
the Coalition for Evidence-Based Policy’s recent brief , agencies can often use 
administrative data (such as data on wages, employment, emergency room visits or 
school attendance) to conduct rigorous evaluations, including evaluations that rely on 
random assignment, at low cost. Similarly, the private sector has used new software and 
online tools to dramatically reduce the time and cost of experimentation. Agencies 
should consider whether they can use such data or technology to support rigorous 
evaluations of their existing programs or new initiatives. 

• Evaluations linked to waivers and performance partnerships: One of the best ways 
to learn about a program is to test variations and subject them to evaluation, using some 
element of random assignment or a scientifically controlled design. OMB invites 
agencies to explain how they will use existing waiver authorities to evaluate different 


approaches to improving outcomes. Agencies should also consider seeking authority 
from Congress, through the FY 2014 budget process, to allow new waivers linked to 
evaluation or to establish cross-agency “performance partnerships” that enable blending 
of multiple funding streams to test better ways to align services and improve outcomes. 
Several agencies are seeking such authority in 20 1 3 for initiatives supporting distressed 
communities and disconnected youth. 

• Expansion of evaluation efforts within existing programs: In addition to specifying 
evaluations to be performed with dedicated funding, agencies can also add a general 
policy and requirements favoring evaluation into existing grants, contracts, or waivers. 
These measures may require new legislation. For example, Congress recently approved 
the Department of Labor’s request for a small cross-agency set-aside for evaluation 
activities. 

• Systemic measurement of costs and cost per outcome: Agencies are encouraged to 
include measurement of costs and costs per outcome as part of the routine reporting of 
funded programs to allow for useful comparison of cost-effectiveness across programs. 

Agencies should release evaluations promptly through either their agency websites or 
alternative means. OMB particularly welcomes agency proposals to improve public 
access to, and understanding of, evidence about what works and what does not. 

2. Using comparative cost-effectiveness data to allocate resources . Through the Pew Charitable 
Trust’s Results First initiative, a dozen States are currently adopting a model developed by 
the Washington State Institute for Public Policy (WSIPP) that ranks programs based on the 
evidence of their return on investment. Once evidence-based programs have been identified, 
such an analysis can improve agency resource allocation and inform public understanding. 
For example, the Environmental Protection Agency and the U.S. Department of Agriculture 
are working together to incorporate evidence about the cost-effectiveness of different 
pollution control strategies in the Chesapeake Bay restoration effort. 

OMB invites agencies to identify areas where research provides strong evidence regarding 
the comparative cost-effectiveness of agency investments. The research may pertain to the 
allocation of funding across agency programs (e.g., research showing that some funding 
streams have higher returns on investments) or within programs (e.g., research showing that 
some types of grantees or programmatic approaches have higher returns). Agencies should 
describe the body of research and then apply its results to support a proposed resource re- 
allocation. OMB is more likely to support an existing resource allocation or a request for 
new resources supported in this way, and may feature the agency’s reasoning in the 2014 
Budget. 

3. Infusing evidence into grant-making . Grant-making agencies should demonstrate that, 
between FY 2013 and FY 2014, they are increasing the use of evidence in formula and 
competitive programs. Agencies should consider the following approaches, among others: 
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• Encouraging use of evidence in formula grants: OMB invites agencies to propose 
ways to increase the use of evidence-based practices within formula grant programs. For 
example, formula funds can be conditioned on the adoption of evidence-based practices, 
and high-quality technical assistance can be used to share and support implementation of 
evidence-based practices. Competitive programs can assign points to applicants based on 
their integration of such practices into formula streams. 

• Evidence-based grants: Several agencies — ranging from the Department of Education 
to the U.S. Agency for International Development — have implemented evidence-based 
grant programs that apply a tiered framework to assess the evidence supporting a 
proposed project and to determine appropriate funding levels. Under this approach, 
programs supported by stronger evidence, as established in a rigorous agency process, are 
eligible for more funding. All programs are expected to evaluate their results. Examples 
of tiered-evidence programs include the Department of Education’s Investing in 
Innovation program and the Department of Health and Human Services’ Teen Pregnancy 
Prevention and Home Visiting programs. 

Even without creating tiers, agencies can provide points or significant competitive 
preference to programs that the agency determines are backed by strong evidence, and 
can build the evidence base by embedding evaluation into programs. Because running 
evidence-based programs requires more resources, agencies may wish to combine 
multiple smaller programs into larger, evidence-based efforts. 

• Pay for Success: Taking the principle of acting on evidence one step further, the 
Departments of Justice and Labor will be inviting grant applicants to use a “pay for 
success” approach, under which philanthropic or private entities (the “investors”) pay 
providers upfront and are only repaid by the government if certain outcomes are met. 
Payment amounts are based, in part, on the amount that the Federal, State, or local 
government saves. A pay-for-success approach is appropriate where: (i) improved 
prevention or other up-front services can produce better outcomes that lead to cost 
savings at the Federal, State, or local level; and (ii) foundations or others are willing to 
invest. 

To date, the Administration has focused its Pay for Success planning on programs 
financed with discretionary appropriations. OMB invites agencies to apply a pay-for- 
success model for programs funded by either discretionary or mandatory appropriations. 
Agencies should also consider using the new authority under the America COMPETES 
legislation to support incentive prizes of up to $50 million. Like Pay for Success, well- 
designed prizes and challenges can yield a very high return on the taxpayer dollar. 

4. Using evidence to inform enforcement . Rigorous evaluation of strategies for enforcing 
criminal, environmental, and workplace safety laws often reveals that some approaches are 
significantly better than others at securing legal compliance. OMB encourages agencies to 
indicate how their allocation or reallocation of resources among enforcement strategies is 
informed by such evidence. 
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5. Strengthening agency evaluation capacity . Agencies should have a high-level official who is 
responsible for program evaluation and can: 

Develop and manage the agency’s research agenda; 

Conduct or oversee rigorous and objective studies; 

Provide independent input to agency policymakers on resource allocation and to 
program leaders on program management; 

Attract and retain talented staff and researchers, including through flexible hiring 
authorities such as the Intergovernmental Personnel Act; and 
Refine program performance measures, in collaboration with program managers and 
the Performance Improvement Officer. 

These goals can be accomplished by different kinds of leaders, ranging from a chief 
evaluation officer who reports to the Secretary or Deputy Secretary to the head of an 
independent institute in the agency. An existing official could play the role, or a forceful 
new position could replace several less empowered ones. OMB invites agencies to propose 
in their budget submissions ways to strengthen the agency’s evaluation capacity, within tight 
resource constraints. 

Support for Evidence-Based Initiatives 

OMB invites your agency to participate in a number of forums to improve use of 
evidence: 

• OMB and the Council of Economic Advisers will organize a series of topical discussions 
with senior policy officials and research experts in the agencies. The meeting agendas 
will focus on administrative and policy levers for driving an increasing share of Federal 
investments into evidence-based practices. We will plan summer meetings in order to 
help inform agencies’ evaluation plans and budget submissions, and will also have 
follow-up meetings in the fall. 

• OMB will reinvigorate the interagency evaluation working group established in 2010 
with a series of meetings focused on issues commonly affecting evaluators, such as 
procurement rules, the Paperwork Reduction Act, and the integration of evidence in 
agencies’ decision-making process. 

• The Performance Improvement Council will convene research, performance 
management, and program officials to develop ways to improve performance measures, 
validate their correlation with outcome data from program impact evaluations, and use 
data analytics to support more cost-effective decision-making. 

• The Office of Science and Technology Policy has created a “community of practice” for 
agency personnel involved in designing and managing incentive prizes and has organized 
a Science of Science Policy working group that is developing tools aimed at establishing 
a more scientific, empirical evidence basis for science and technology policymaking. 
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To discuss which ideas in this memo make most sense at your agency, please contact 
your agency’s OMB contact. For more general support on evidence-based policy and evaluation, 
you also may contact Dan Rosenbaum ('Dan T. Rosenbaum@omb.eop.iiov ). 
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